Recognise direct buffers in heap size docs #42070
Conversation
This commit slightly reworks the recommendations in the docs about setting the heap size: * the "rules of thumb" are actually instructions that should be followed * the reason for setting `Xmx` to 50% of the heap size is more subtle than just leaving space for the filesystem cache * it is normal to see Elasticsearch using more memory than `Xmx` * replace `cutoff` and `limit` with `threshold` since all three terms are used interchangeably * since we recommend setting `Xmx` equal to `Xms`, avoid talking about setting `Xmx` in isolation Relates elastic#41954
DaveCTurner
added
>docs
|
Pinging @elastic/es-core-infra |
|
Build failed for mysterious possibly-infrastructural reasons. @elasticmachine please run elasticsearch-ci/docbldesx. |
|
|
||
| heap address: 0x0000000118400000, size: 28672 MB, Compressed Oops with base: 0x00000001183ff000 | ||
| -- | ||
|
|
||
| The more heap available to {es}, the more memory it can use for its internal | ||
| caches, but the less memory it leaves available for the operating system to use | ||
| for the filesystem cache. Also, larger heaps can sometimes cause longer garbage |
There was a problem hiding this comment.
I think we should leave "sometimes" out here (the sentence is already indefinite by saying "can" and "sometimes" makes the sentence too weak IMO).
|
@elasticmachine run elasticsearch-ci/docbldesx |
This commit slightly reworks the recommendations in the docs about setting the heap size: * the "rules of thumb" are actually instructions that should be followed * the reason for setting `Xmx` to 50% of the heap size is more subtle than just leaving space for the filesystem cache * it is normal to see Elasticsearch using more memory than `Xmx` * replace `cutoff` and `limit` with `threshold` since all three terms are used interchangeably * since we recommend setting `Xmx` equal to `Xms`, avoid talking about setting `Xmx` in isolation Relates #41954
* elastic/master: (84 commits) [ML] adds geo_centroid aggregation support to data frames (elastic#42088) Add documentation for calendar/fixed intervals (elastic#41919) Remove global checkpoint assertion in peer recovery (elastic#41987) Don't create tempdir for cli scripts (elastic#41913) Fix debian-8 update (elastic#42056) Cleanup plugin bin directories (elastic#41907) Prevent order being lost for _nodes API filters (elastic#42045) Change IndexAnalyzers default analyzer access (elastic#42011) Remove reference to fs.data.spins in docs Mute failing AsyncTwoPhaseIndexerTests Remove close method in PageCacheRecycler/Recycler (elastic#41917) [ML] adding pivot.max_search_page_size option for setting paging size (elastic#41920) Docs: Tweak list formatting Simplify handling of keyword field normalizers (elastic#42002) [ML] properly nesting objects in document source (elastic#41901) Remove extra `ms` from log message (elastic#42068) Increase the sample space for random inner hits name generator (elastic#42057) Recognise direct buffers in heap size docs (elastic#42070) shouldRollGeneration should execute under read lock (elastic#41696) Wait for active shard after close in mixed cluster (elastic#42029) ...
This commit slightly reworks the recommendations in the docs about setting the heap size: * the "rules of thumb" are actually instructions that should be followed * the reason for setting `Xmx` to 50% of the heap size is more subtle than just leaving space for the filesystem cache * it is normal to see Elasticsearch using more memory than `Xmx` * replace `cutoff` and `limit` with `threshold` since all three terms are used interchangeably * since we recommend setting `Xmx` equal to `Xms`, avoid talking about setting `Xmx` in isolation Relates elastic#41954
|
@DaveCTurner @jasontedor thank you for this! When I read the diff of the docs, it made me feel like there was a difference in guidance about setting Xms and Xmx to the same value. However, reading the commit message, I see that it was just text-reorganization. Can you remind me the reasons and severity of "should" in Is it "strongly should" in the typical deployment of Elasticsearch, where it's the only / the dominant application on a particular machine; but more flexible in more complex shared-usage situations? Or is the intensity of "should" the same for multi-tenant and dedicated deployments both? |
|
@cjcjameson would you ask this question over on the discussion forum? It's a good question, but this isn't the right place to answer it. |
This commit slightly reworks the recommendations in the docs about setting the
heap size:
the "rules of thumb" are actually instructions that should be followed
the reason for setting
Xmxto 50% of the heap size is more subtle than justleaving space for the filesystem cache
it is normal to see Elasticsearch using more memory than
Xmxreplace
cutoffandlimitwiththresholdsince all three terms are usedinterchangeably
since we recommend setting
Xmxequal toXms, avoid talking about settingXmxin isolationRelates #41954